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Abstract 


This study examined the relationship of community college programs and services to retention of 
students in four community colleges, with an emphasis on determining whether outcomes vary 
for students in occupational programs and how student characteristics moderate these effects, 
with the goal of determining what is correlated with success. Overall, the study found evidence 
for positive impacts resulting from having an occupational major; receiving most types of 
financial aid; having higher placement scores, particularly in math; taking developmental math; 
accessing tutoring services in the first tenn; and choosing an occupational major. White students 
were more likely than others to be retained and students who experienced multiple stressful life 
events, or who worried about paying tuition, were less likely to be retained. It was more useful 
for students to complete all the credits in which they enrolled in the first term than to enroll full- 
time in the first tenn. 



Relative Impact of Interventions to Improve Achievement and Retention 
in Postsecondary Occupational Programs 


Community colleges are considered a more nurturing environment than many four-year 
institutions and are perceived by students as supportive of learning (Carlan & Byxbe, 2000). 
However, completion rates for students at community colleges are unacceptably low. Despite the 
apparent benefits to students of community college degrees and credentials, fewer than half of 
students intending to complete an Associate degree do so within five years; outcomes are only 
slightly better for those working toward non-degree sub-baccalaureate credentials or certificates 
(Bailey, Alfonso, Scott, & Leinbach, 2004). 

The Relative Impact study was a longitudinal correlational study using institutional data from 
four U.S. public two-year colleges in four different states that had occupational 1 programs and 
used common programs and services aimed at improving student retention and completion and 
that could be analyzed at the student level. Two cohorts of students were included in the study: 
those who entered college in Fall 2009 and Fall 2010, respectively. The research team initially 
hoped to examine the relationship between a large number of interventions and combinations of 
programs and services with student retention, but sufficient data for analysis were available only 
for a smaller number, as described below. Although many previous studies have studied 
individual programs and services at a single college, the Relative Impact study examined these 
multiple programs and services as they were implemented by multiple colleges and experienced 
by students throughout two to three years of enrollment. The primary audiences for this study are 
community college administrators, instructors, and retention specialists; postsecondary 
researchers and funders; and state policymakers. 

The study measured associations among student characteristics and common programs and 
services (developmental English, reading, and writing courses; developmental math courses; 
financial aid; and tutoring) and student outcomes, specifically retention, completion, and grade 
point average (GPA). In addition, the study aimed to identify programs and services correlated 
with two intennediate outcomes: (a) the ratio of credits attempted to credits earned and (b) grade 
point average (GPA). A further objective was to determine the degree to which the programs and 
services were related to positive outcomes for students differing by age, sex, race, occupational 
versus non-occupational major (as determined by the individual colleges based on factors such as 
degree program and declared intent of enrollment), stress as reported in Tenn 1 by Cohort 2, the 
ratio of credits earned to credits attempted in the first tenn, and enrollment intensity in the first 
term (full-time vs. part-time enrollment). 

Research Question. Our primary research question was: What programs and sendees and 
combinations of programs and services are most strongly associated with retention and 
completion for students in postsecondary programs, and how do student background and other 
characteristics moderate those associations? 


1 This study uses the term “occupational” to refer to programs and students for which colleges use terms such as 
career and technical, vocational-technical, and professional and technical. Typically, these are programs that 
culminate in a less-than-four-year credential that is intended to lead to employment. Colleges vary to a small degree 
in how they classify some programs. 
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Rationale and Literature Review 


The nation’s community colleges play a critical role in providing students with access to 
affordable postsecondary education. Completion of a community college degree or credential can 
provide a path to a career or to further education. However, only about one-third of students who 
enroll in a community college complete a credential (Bailey & Alfonso, 2005). The causes 
underlying this statistic are complex. In an ACT policy report reviewing hundreds of studies of 
retention at four-year institutions, Lotkowski, Robbins, and Noeth (2004) found that both 
academic and non-academic factors were predictive of retention; among these were 
socioeconomic status, high school GPA, ACT assessment scores, academic self-confidence, 
academic goals, social support, financial support, social involvement, and institutional 
commitment. A similar review of two-year college retention is not available in the literature. 

Both policymakers and practitioners have noted that the traditional metrics of institutional 
effectiveness — graduation or completion rates of entering degree-seeking full-time students — 
apply more aptly to four-year institutions and fail to acknowledge the multiple missions of 
community colleges (Bailey, Leinbach, & Jenkins, 2005). Recent efforts to establish additional 
or alternative measures by the federal Committee on Measures of Student Success (Higher 
Education Opportunity Act of 2008) and by the American Association of Community Colleges 
(2011b), through its participation in the Voluntary Framework of Accountability, reflect a shift 
toward acknowledging the distinctive nature of community college students. With respect to the 
Relative Impact study, it should be noted that the study’s data collection timeframe was too short 
to expect large numbers of students to graduate; thus our focus was primarily on retention. 

Colleges could better serve students by understanding the mediating and moderating effects of 
factors such as age, placement scores, and receipt of financial aid, among others. A better 
understanding of the departure process and the relative effectiveness of programs and services 
may result in (a) more effective programs and services that help students attain their educational 
goals and (b) conceptual models with greater explanatory and predictive power. 

Community college completion and workforce preparation. Community and technical 
colleges help students move into high-skill, high-wage, and high-demand occupations. 

According to the National Center for Education Statistics (2008), in 2004-2005, 64% of students 
seeking associate’s degrees and 81% of students in certificate programs majored in occupational 
fields. Occupational programs at two-year public colleges play a critical role in reducing poverty 
among individuals and families (Hollenbeck & Huang, 2003; Kane & Rouse, 1995). Previous 
research has shown that when students complete their programs, they improve their economic 
circumstances (Hollenbeck & Huang, 2003; Mathur, Reichle, Strawn, & Wiseley, 2004; Prince 
& Jenkins, 2005). However, among community college students who enroll with the goal of 
earning an Associate degree or certificate, fewer than half actually complete a credential of any 
kind (Silverberg, Warner, Fong, & Goodwin, 2004). Those students who take courses without 
earning a credential receive little to no economic return (Grubb, 2002). Colleges are thus paying 
increased attention to retention, as the negative consequences of non-completion become more 
apparent (Upcraft, Gardner, & Barefoot, 2005). 

Despite the clear benefits of completing community college programs, Bailey et al. (2004) found 
that more than two-thirds of occupational majors at sub-baccalaureate institutions left after 
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having completed a year or less of coursework over a five-year period. Their study examined 
completion rates of certificate, Associate, and baccalaureate degree seekers. Associate 
occupational degree seekers were less likely to complete their programs than either baccalaureate 
or certificate seekers. Bailey et al. listed some of the factors that may contribute to this 
differential success, such as the shorter period of time that it takes to complete most certificate 
programs (usually one year), and the delayed, part-time, or interrupted enrollment patterns 
common among Associate degree seekers. 

Although some students leave college for employment in their field of study, low GPA (Metzner 
& Bean, 1987) and uncertainty about career choice (Pascarella & Terenzini, 1991) have been 
found to be among the most powerful predictors of early college departure. Other barriers to 
program completion include inadequate pre-college academic skills, insufficient goal 
commitment, limited social integration, financial pressures, family responsibilities, transportation 
problems, health concerns, and disability (Astin & Oseguera, 2003; Bray, Braxton, & Sullivan, 
1999; Cabrera, Nora, & Castaneda, 1992; Cofer & Somers, 1999; Pascarella & Terenzini, 1991; 
Schlossberg, Lynch, & Chickering, 1989; St. John, Kirschstein, & Noell, 1991). Any or all of 
these barriers can stand in the way of engagement (integration), achievement, and successful 
transition into and through postsecondary education to employment (Pascarella & Terenzini, 
1991,2005). 

To be sure, not all entering community college students intend to complete a degree or 
credential. Some students attend community college to maintain professional licensure or 
certification, gain supplementary job skills, or engage in personal enrichment, for example. 
However, many students enter community colleges with a degree or credential goal in mind, and 
many of these students never attain that goal. This group of students concerned us most. We 
collected data on student intent and major in an effort to identify as clearly as possible those 
students who planned to earn a degree or credential and those who planned to transfer to another 
institution. 

Practical importance of the variables. Theory and research suggest that different programs and 
services will be more or less effective for individuals from different demographic groups and 
with different characteristics or contextual influences, but these mediating and moderating 
factors are not well understood. Keith (2007) found relationships between students’ use of 
support services and their prior educational experiences, institutional barriers (e.g., class 
availability, class times), and situational barriers (e.g., marital status, employment status, age). 
For example, nontraditional students with characteristics more similar to their traditional-age 
peers (i.e., those who were younger, unemployed, and had experienced stress from increased 
tuition) used more academic services than other nontraditional students at the same institution. 
Likewise, a random assignment study of the Beacon Mentoring program at South Texas College 
(MDRC, 2010) found that: 

While the program has not improved math class pass rates or persistence in college 
overall, it benefited two subgroups at particular risk of failure: (1) part-time students were 
less likely to withdraw from and more likely to pass the math class, earned more credits, 
and, at least in the developmental math classes, scored higher on the final exam, and (2) 
developmental student were less likely to withdraw from math class than students in the 
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control group, and they earned more credits in their non-math developmental courses. 
(Bringing Student Services into the Classroom, para. 2) 

Students in community college programs have a wide range of backgrounds (American 
Association of Community Colleges, 2011a). For example, less than half (43%) of community 
college students are under the age of 21, which is often considered the “traditional” college age. 
Nationally, 35% of community college students are from racial or ethnic minority groups; this 
figure is higher at urban community colleges. Nearly 60% of community college students attend 
college less than full-time. Nearly 80% of community college students work either full- or part- 
time, and 50% work full-time. Many community college students (39%) are the first in their 
families to attend college, and many have additional responsibilities such as caring for children 
or elders (e.g., 17% are single parents). Thirteen percent of postsecondary students report that 
English is not the primary language spoken at home (Schuck & Larson, 2003). The percentage of 
youth with disabilities participating in postsecondary education more than doubled between 1987 
and 2003, from 15% to 32% (Newman, 2005). In addition (42%) of students at two-year public 
institutions reported taking remedial coursework at some point to make up for their lack of 
academic preparation for college-level coursework. 

Intervention programs. Evidence for the effectiveness of retention programs and services 
included in this study was reviewed in a prior report (Bremer et al., 2011). The programs and 
services that were ultimately included in our analyses were chosen based on the availability of 
sufficient data from our sites. Data for some variables were not available for all sites. 

Conceptual Model 

Figure 1 is a conceptual model of the Relative Impact study, including variables affecting student 
retention and related outcomes. This model provides a framework for an input-process-output 
model of student experiences and outcomes. 
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Freedom 

County 

College 


River 

Port 

College 


Scenic 

Hills 

College 


Valley 

West 

College 


Student Demographics 

(Non-malleable) 

• Race (White/non-White) 

• Gender (M/F) 

• Age ( continuous or 
dichotomous: over 
25/25 or younger) 

Student Characteristics 

(Malleable) 

• Placement test scores 

• Intensity of enrollment 
(full-time or part-time) 
(in first term) 

• Ratio of credits earned 
to credits attempted (in 
first term) 

• CTE/Occupational 
major (ever) 

• Participation in 
programs and services 
o financial aid (ever) 

■ Pell grant 

■ Stafford 
subsidized loan 

■ Stafford 
unsubsidized 
loan 

* Other 
o developmental 
coursework (in the 
first term) 

o tutoring (in the first 
term) 

• Stressors (Cohort 2 
only) (at time of survey) 


Proximal 


Distal 

Outcomes 


Outcomes 

• GPA 


• Retention 

(cumulative) 


• Completion 


FIGURE 1 . Conceptual model of the Relative Impact study. Note. Cohort 1 is comprised of all 
students who entered one of the colleges for the first time in Fall 2009; Cohort 2 is comprised of 
all students who entered one of the colleges for the first time in Fall 2010. Occupational (Occ) 
status of students was determined by each college based on factors such as degree program and 
declared intent. Site names are pseudonyms. 
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Tables 1.1, 1.2, and 1.3 show descriptive statistics for the cohorts by site. 


Table 1.1: Descripi 

tive Statistics - Total Counts 

Site 

Cohort 1 

Cohort 2 

Total 

Count 

% 

Count 

% 

Count 

% 

Freedom County 

1,720 

30.3 

1,896 

32.4 

3,616 

31.4 

River Port 

1,963 

34.6 

2,096 

35.9 

4,059 

35.3 

Scenic Hills 

905 

16.0 

915 

15.7 

1,820 

15.8 

Valley West 

1,081 

19.1 

938 

16.0 

2,019 

17.5 

Total 

5,669 

100.0 

5,845 

100.0 

11,514 

100.0 


Table 1.2: Descriptive Statistics - Continuous Variables By Site 

Continuous Variables 

Freedom County 

M 

SD 

Valid N 

Missing 

Valid N% 

Age 

23 

7 

3,616 

0 

100 

Computed cumulative GPA 

1.96 

1.25 

3,592 

24 

99 

Standardized placement test reading 

.00 

1.00 

2,584 

1,032 

71 

Standardized placement test writing 

.00 

1.00 

24 

3,592 

1 

Standardized placement test math 

.00 

1.00 

3,303 

313 

91 


Continuous Variables 

River Port 

M 

SD 

Valid N 

Missing 

Valid N% 

Age 

28 

10 

3,890 

169 

96 

Computed cumulative GPA 

2.45 

1.26 

3,699 

360 

91 

Standardized placement test reading 

.00 

1.00 

3,198 

861 

79 

Standardized placement test writing 

.00 

1.00 

2,434 

1,625 

60 

Standardized placement test math 

.00 

1.00 

3,202 

857 

79 


Continuous Variables 

Scenic Hills 

M 

SD 

Valid N 

Missing 

Valid N% 

Age 

26 

10 

1,820 

0 

100 

Computed cumulative GPA 

2.50 

1.12 

1,624 

196 

89 

Standardized placement test reading 

.04 

.97 

1,125 

695 

62 

Standardized placement test writing 

.06 

.97 

1,135 

685 

62 

Standardized placement test math 

.01 

1.00 

1,184 

636 

65 


Continuous Variables 

Valley West 

M 

SD 

Valid N 

Missing 

Valid N% 

Age 

25 

10 

2,019 

0 

100 

Computed cumulative GPA 

2.70 

1.03 

1,802 

217 

89 

Standardized placement test reading 

.00 

1.00 

1,940 

79 

96 

Standardized placement test writing 

.00 

1.00 

1,940 

79 

96 

Standardized placement test math 

.00 

1.00 

1,935 

84 

96 
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Continuous Variables 

All Sites 

M 

SD 

Valid N 

Missing 

Valid A % 

Age 

25 

9 

11,345 

169 

99 

Computed cumulative GPA 

2.34 

1.23 

10,717 

797 

93 

Standardized placement test reading 

.01 

1.00 

8,847 

2,667 

77 

Standardized placement test writing 

.01 

.99 

5,533 

5,981 

48 

Standardized placement test math 

.00 

1.00 

9,624 

1,890 

84 
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Table 1.3: Categorical Variables 

Categorical Variables 

Site 

Freedom County 

River Port 

Scenic Hills 

Valley West 

Total 

Count 

% 

Count 

% 

Count 

% 

Count 

% 

Count 

% 

Student Demographics 

Female 

Male 

1,704 

47.1 

1,795 

44.2 

951 

52.3 

920 

45.6 

5,370 

46.6 

Female 

1,866 

51.6 

2,089 

51.5 

869 

47.7 

1,097 

54.3 

5,921 

51.4 

Subtotal 

3,570 

98.7 

3,884 

95.7 

1,820 

100.0 

2,017 

99.9 

11,291 

98.1 

Missing 

46 

1.3 

175 

4.3 

0 

0.0 

2 

.1 

223 

1.9 

White 

No 

572 

15.8 

2,136 

52.6 

398 

21.9 

377 

18.7 

3,483 

30.3 

Yes 

2,899 

80.2 

1,668 

41.1 

1,387 

76.2 

1,536 

76.1 

7490 

65.1 

Subtotal 

3,471 

96.0 

3,804 

93.7 

1,785 

98.1 

1,913 

94.7 

10,973 

95.3 

Missing 

145 

4.0 

255 

6.3 

35 

1.9 

106 

5.3 

541 

4.7 

Over 25 
years old 

25 or under 

2,847 

78.7 

2,189 

53.9 

1,248 

68.6 

1,402 

69.4 

7,686 

66.8 

over 25 

769 

21.3 

1,699 

41.9 

572 

31.4 

617 

30.6 

3,657 

31.8 

Subtotal 

3,616 

100.0 

3,888 

95.8 

1,820 

100.0 

2,019 

100.0 

11,343 

98.5 

Missing 

0 

0.0 

171 

4.2 

0 

0.0 

0 

0.0 

171 

1.5 

Student Characteristics 

Attempted 12 
or more 

credits 

No 

609 

16.8 

1,874 

46.2 

705 

38.7 

862 

42.7 

4,050 

35.2 

Yes 

2,957 

81.8 

2,185 

53.8 

1,115 

61.3 

1,157 

57.3 

7,414 

64.4 

Subtotal 

3,566 

98.6 

4,059 

100.0 

1,820 

100.0 

2,019 

100.0 

11,464 

99.6 

Missing 

50 

1.4 

0 

0.0 

0 

0.0 

0 

0.0 

50 

.4 

Enrolled in 
DevEd 

English, 
reading, or 
writing in the 
1st term 

No 

2,918 

80.7 

3,196 

78.7 

1,566 

86.0 

1,529 

75.7 

9,209 

80.0 

Yes 

698 

19.3 

863 

21.3 

254 

14.0 

490 

24.3 

2,305 

20.0 

Subtotal 

3,616 

100.0 

4,059 

100.0 

1,820 

100.0 

2,019 

100.0 

11,514 

100.0 

Enrolled in 
DevEd math 

No 

2,639 

73.0 

3,405 

83.9 

1,244 

68.4 

893 

44.2 

8,181 

71.1 

Yes 

977 

27.0 

654 

16.1 

576 

31.6 

1,126 

55.8 

3,333 

28.9 



Received 
Financial 
aids ever 

No 

702 

19.4 

842 

20.7 

1,077 

59.2 

587 

29.1 

3,208 

27.9 

Yes 

2,914 

80.6 

3,217 

79.3 

743 

40.8 

1,432 

70.9 

8,306 

72.1 

Subtotal 

3,616 

100.0 

4,059 

100.0 

1,820 

100.0 

2,019 

100.0 

11,514 

100.0 

Occupational 
status ever 

Non-Occ 

634 

17.5 

1,042 

25.7 

924 

50.8 

1,118 

55.4 

3,718 

32.3 

Occ 

2,982 

82.5 

3,017 

74.3 

896 

49.2 

901 

44.6 

7,796 

67.7 

Subtotal 

3,616 

100.0 

4,059 

100.0 

1,820 

100.0 

2,019 

100.0 

11,514 

100.0 

Visited 
tutoring 
center or not 
in the 1 st 

term 

No 

3,426 

94.7 

3,759 

92.6 

0 

0.0 

1,592 

78.9 

8,777 

76.2 

Yes 

190 

5.3 

300 

7.4 

0 

0.0 

427 

21.1 

917 

8.0 

Missing 

0 

0.0 

0 

0.0 

1,820 

100.0 

0 

0.0 

1,820 

15.8 

Subtotal 

3616 

100.0 

4,059 

100.0 

1,820 

100.0 

2,019 

100.0 

11,514 

100.0 

Student Outcomes 










Enrolled in 
the 2nd term 

No 

674 

18.6 

1,144 

28.2 

549 

30.2 

469 

23.2 

2,836 

24.6 

Yes 

2,918 

80.7 

2,915 

71.8 

1,271 

69.8 

1550 

76.8 

8,654 

75.2 

Missing 

24 

.7 

0 

0.0 

0 

0.0 

0 

0.0 

24 

.2 

Subtotal 

3,616 

100.0 

4,059 

100.0 

1,820 

100.0 

2,019 

100.0 

11,514 

100.0 

Enrolled in 
the 1st year 

No 

1,595 

44.1 

2,317 

57.1 

1,042 

57.3 

1027 

50.9 

5,981 

51.9 

Yes 

1,997 

55.2 

1,742 

42.9 

778 

42.7 

992 

49.1 

5,509 

47.8 

Missing 

24 

.7 

0 

0.0 

0 

0.0 

0 

0.0 

24 

.2 

Subtotal 

3,616 

100.0 

4,059 

100.0 

1,820 

100.0 

2,019 

100.0 

11,514 

100.0 

Enrolled or 
not in the 2nd 
year 

No 

2,205 

61.0 

1,486 

36.6 

713 

39.2 

732 

36.3 

5,136 

44.6 

Yes 

1,387 

38.4 

477 

11.8 

192 

10.5 

349 

17.3 

2,405 

20.9 

Missing 

24 

.7 

2,096 

51.6 

915 

50.3 

938 

46.5 

3,973 

34.5 

Subtotal 

3,616 

100.0 

4,059 

100.0 

1,820 

100.0 

2,019 

100.0 

11,514 

100.0 

Graduated 

ever 

No 

3,422 

94.6 

3,534 

87.1 

1,663 

91.4 

1,961 

97.1 

10,580 

91.9 

Yes 

194 

5.4 

525 

12.9 

157 

8.6 

58 

2.9 

934 

8.1 

Subtotal 

3,616 

100.0 

4,059 

100.0 

1,820 

100.0 

2,019 

100.0 

11,514 

100.0 
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Methods 

I. Research Design 

The Relative Impact study made use of a longitudinal quasi-experimental design in which data 
were obtained from the institutional databases of two-year community colleges participating in 
the study. Given that the goal was to identify the programs and services that are associated with 
retention and completion for students in postsecondary programs, as well as the interactions of 
these with other variables, we worked to statistically control for variables that might have 
confounded the relationship between participation in the programs and services and retention and 
completion. 

Design and Procedure 

The research protocol was approved by the Institutional Review Board at the University of 
Minnesota, as well as the associated institutional staff and respective IRB boards, if available, at 
each college. Education records were acquired in accordance with Family Educational Rights 
and Privacy Act guidelines. Data management specialists at each college were asked to provide 
data for the two cohorts following the close of each term, beginning with Fall 2009 and ending in 
Spring 2012. 

Instrumentation and Data Preparation 

Data provided by the four college sites varied in availability and completeness. For many 
categories, the data included educational records of equivalent content but differing coding 
schemes across colleges. 

Enrollment in Developmental Education Courses. Each college provided course enrollment 
records, associated with students’ research study ID numbers, and a list of developmental 
courses. Three dummy-coded variables were computed at each college, one for enrollment in 
developmental math (DM) and one for enrollment in development English, reading, or writing 
(DERW), and a third for students who took both DERW and DM courses. Further, the number of 
DERW courses and the number of DM courses taken in the first term was calculated. 

Retention. Three retention outcomes were computed: immediate retention (i.e., enrollment in the 
second term), one-year retention (i.e., enrollment in the second fall), and two-year retention (i.e., 
enrollment in the third fall). Note that only Cohort 1 had data available with regard to two-year 
retention. 

Grade point average. Grade point average (GPA) was calculated using algorithms provided by 
each college; for example, one college granted 4 grade points for an A-, whereas another granted 
3.7 grade points. Some courses differed in whether or not they counted toward GPA calculation, 
such as courses taken as pass/fail. These were not included in our GPA calculations. 

Three permutations of GPA were computed. First, an overall cumulative GPA was computed, 
combining academic performance across all enrolled terms. Next, GPA was computed for 
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academic performance in two specific subject areas: (a) non-developmental college-level courses 
in English, reading, or writing and (b) non-developmental college-level courses in math. 

Completion. A student was considered to have graduated if she or he completed an Associate 
degree, if they were documented as transferring to another institution, or if they completed an 
occupational certificate offered by the college. Certificates, which typically require one year or 
less of full-time enrollment, demonstrate to would-be employers that the student has adequate 
training in a particular field. Although we hoped to include completion data concerning the 
awarding of nationally recognized industry credentials, such as the NCLEX (National Council 
Licensure Examination) nursing credential, insufficient data were available from the colleges to 
include these in the analyses. Outcomes for students who exited a college before completing a 
degree or certificate but returned in a later term during the course of this study (i.e., stopped out) 
were included. 

Intermediate outcomes. Data were collected on course grades in non-developmental English, 
reading, or writing and math courses. 

Institutions. Dummy variables were created to indicate the four different colleges. These were 
used as covariates in the analyses to account for variation in outcomes due to institutional 
uniqueness, such as the method used for computing GPA at each school. 

Demographics. Age was at the time of enrollment. Records on ethnicity provided a number of 
categories that were reduced to either White/non-Hispanic or Other, due to low numbers of 
students from individual minority populations. Sex was limited to male or female. 

Occupational status. Occupational status refers to whether students had selected a program in a 
career and technical education (CTE) field. These are programs that aim to make students 
employable after completion; credits may or may not transfer to a baccalaureate program. Non- 
occupational students, often referred to as transfer students, are those whose aim at the two-year 
college is to complete coursework that will count toward a baccalaureate degree. Either the 
educational records provided occupational status, or definitions using intent codes at a college 
were used to compute a dummy variable indicating whether a student had ever been defined as 
occupational at any time throughout his or her enrollment at college 

Financial aid. Educational records contained a wide array of information about financial aid. 

The information provided was reduced to dummy variables that noted receipt of financial aid at 
any time throughout a student’s enrollment at college in each of the following categories: (1) 
Federal Pell grants, (2) subsidized Stafford loans, (3) unsubsidized Stafford loans, and (4) other 
financial aid. 

Tutoring. A dummy variable was created to indicate a student’s use of tutoring services within 
the first tenn of enrollment. 

Placement tests. The educational records contained placement testing data from the ASSET, 
ACCUPLACER, or COMPASS exam brands. Because the three brands of test measure similar 
content, and because there is no reason to believe the ability of the students taking these tests 
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would systematically differ across the three, standardized scores for each student on the tests 
they did take were created. That is, each student received a standardized score for reading, 
writing, and math, regardless of which brand of test they took. Where students took a higher or 
lower level math placement test, scores were adjusted for the difficulty of the test. These 
adjustments were based on the mean differences on the test scores, among those students who 
took more than one level of placement test. (This was common enough to ascertain a rough test 
equivalent.) Note that these tests are only approximate guides to a student’s ability, only valid at 
the time that the test was taken, and that the equivalences are approximate at best. Even so, these 
scores provide a rough indication of a student’s abilities in English, writing, and math at the time 
of entering college. 

II. Sample 

Site Selection and Characteristics. Site selection for this study was guided by a process of focal 
sampling (Anastas, 1999), in which participating community college sites or subjects were 
selected “not to approximate representativeness but because they are atypical in some way that 
specially equips them to be useful as study informants” (p. 288). For this study, the sites sought 
were atypical in that it was necessary to find institutions with detailed data collection procedures 
in place, in addition to having programs and services of interest and a strong commitment on the 
part of administrators to participate in the study. 

To identify such sites, the project staff turned to members of a U.S. Department of Education, 
Office of Vocational and Adult Education-nominated expert panel of state-level officials and 
representatives from workforce development organizations. This panel was originally convened 
in 2006 for a previous NRCCTE study on career pathway programs (Bragg et ah, 2007). In 
addition to drawing upon the recommendations of these panel members, the site selection 
process also included contacting experts from states, colleges, and organizations known for their 
high-quality work in the field. In some states, the state official who was contacted disseminated 
study information to all community colleges in the state. In others, experts nominated specific 
community colleges they viewed as well-regarded, having with strong retention programs and 
services, and having the kind of data sought. In addition, we identified state systems with strong 
data collection practices and requested nominations from administrators in those states. All four 
selected sites met all of the following criteria: 

• The college was fully accredited. 

• At least four programs and services expected to affect retention (based on the literature) 
were available to students (e.g., financial aid, tutoring, developmental coursework, 
advising). 

• The college had an existing student record database that allowed for the de-identification 
of data and that included frequency data or other measures of intensity concerning 
student participation in programs and services. 

• College institutional research personnel were willing and able to provide the data to the 
researchers. 

• College administrators and staff were highly committed to participating in the study. 
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In addition, three of four sites met the following criterion: 

• The college’s entering student body for Fall 2009 was comprised of at least 10% minority 
students. 

After completing the initial site identification process, we visited each site during Summer 2009 
to ensure that (a) the site would be able to meet the study’s data provision requirements for 
access, quality, and transferability, and (b) that the programs and services claimed by each site 
were in place and accessible to students. A site visit in each subsequent year allowed us to update 
information about current intervention design and current data availability. 

Enrollment. The four sites enrolled a total of 5,674 incoming students in Fall 2009 and 5,733 
students in Fall 2010 who were identified as fitting the study’s parameters. Participants at each 
college were selected by a number of inclusion and exclusion criteria. First, two cohorts of 
students were included in the study: those who entered college in Fall 2009 and Fall 2010. To be 
included in either of these cohorts, students needed to be at least 1 8 years old and enrolled in one 
or more courses in their first term of enrollment. Also included were students whose chosen 
postsecondary program was designed to include one or more initial courses taken during the 
summer prior to Fall 2009 or Fall 2010. Students who indicated that they did not have any 
intention of pursuing a degree or certificate (e.g., they were taking courses for personal 
enrichment) were excluded, as were students attending the college while still in high school and 
students attending Adult Basic Education or GED classes at the college. Also excluded were any 
students who at the time of enrollment specifically declared a non-completion, non-transfer 
intent for taking courses (such as “personal enrichment”). Overall, the sample of students ranged 
in age from 18 to 73, with a mean age of 26.60 ( SD = 10.07). The sample had a slightly greater 
proportion of females (51.3%) than males (46.4%; 2.2% unreported or missing) and a greater 
proportion of individuals of White/non-Hispanic ethnicity (58.1%) compared to other ethnicities 
(36.9%; 5.0% missing or unreported). All Cohort 1 and 2 students were followed using 
institutional data. 

To assess the adequacy of the sample size obtained by this project, the necessary sample size was 
determined using GPower 3.1.3, a statistical power calculation program (Faul, Erdfelder, Lang, 

& Buchner, 2007). Because the outcomes of interest (retention and completion) are dichotomous 
outcomes, an odds ratio was used as the effect size (within the context of a logistic regression) 
for calculation of sample size. Within the context of logistic regression, an odds ratio of 1.5 is 
typically considered a small effect size (Cohen, 1988). However we chose the even more 
conservative value of 1.3. The value of 1.3 is consistent with studies assessing the impact of 
programs and services on retention for at-risk postsecondary students as reported by Valentine et 
al. (2011). The estimate was conditioned on a value of .80 for statistical power, as well as an A" 
value of .15 for the other covariates in the model. Finally, the distribution of the covariate of 
interest (intervention) was set to binomial (as these covariates represent a dichotomous variable: 
student participated/did not participate) with a probability of .2, as this reflects the lowest 
probability of participation across the programs and services observed in our sample. The sample 
size necessary to achieve statistical power of .80 under the above conditions is 4,772, suggesting 
that we had a large enough sample to detect a relatively small effect size for the programs and 
services of interest when at least two of our colleges provided the intervention of interest. Figure 
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2 presents a plot of statistical power as a function of sample size under these conditions. 



III. Data Sources 

Data sources included institutional data, student surveys, student and staff interviews, and 
publicly available sources such as course catalogs and websites. Institutional data were collected 
on two incoming cohorts of community college students (all entering students in Fall 2009 and 
Fall 2010, including former dual enrollment students who exited high school and subsequently 
enrolled in college) at four sites over a three-year period (Fall 2009 through Spring 2012). 

Institutional data. We collected available institutional data from all sites on several variables of 
interest for the two cohorts and received available data on demographic variables (e.g., age, sex, 
ethnicity), enrollment status, placement test scores, financial aid, registration with disability 
services, credits attempted, credits earned, grades, and participation in programs and services. If 
there was no record of a student’s participation in an intervention, it was assumed that the 
student did not participate. The institutional database had missing data to the extent that some 
types of program participation data were simply not collected by the institution or students did 
not fill out some optional fields on their applications. Data were most complete for those items 
already in each college’s main database, such as demographic data, courses taken, grades, and 
receipt of financial aid. Initially, as indicated by prior reports on this study, we had hoped to 
include several more programs and services in the study (e.g., learning communities, student 
success courses, participation in student activities). However, the lack of some programs and 
services (e.g., federal TRIO programs) at some sites, and the lack of sufficient data across sites 
for other programs and services (e.g., student activity participation), limited our ability to 
consider the full range of programs and services thought to aid retention and completion. 

Student survey data and methodology. We collected survey data from samples of students in 
both of the cohorts being studied in order to contextualize and supplement the institutional data. 
Specifically, the purpose of the surveys was to provide a more detailed account of student 
backgrounds, barriers to learning, and experiences with the programs and services offered by 
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each institution (Upcraft et al, 2005). 

Survey questions covered variables that have been identified as affecting retention but which are 
likely to be known only by the students themselves, such as social support for the student’s 
decision to attend college, participation in co-curricular activities, satisfaction with course 
availability, informal contact with faculty (Gerdes & Mallinckrodt, 1994), personal commitment 
to obtaining a credential (Lotkowski et ah, 2004), sense of belonging in the institution (Karp, 
Hughes, & O’Gara, 2008), , and academic self-confidence (Lotkowski et ah, 2004). However, 
the range of responses to some questions provided an inadequate basis for statistical analysis. 

Samples of Cohort 1 students at all four colleges were surveyed twice each year in 2009-2010 
and 2010-2011. A stratified random sample of 200 students was identified, with 55 additional 
students added to replace non-respondents at each site. Within samples, there was oversampling 
of students who declared an occupational major, with the samples consisting of at least two- 
thirds occupational students. The initial survey, sent in February and March 2010, asked about 
these students’ experiences during the Fall 2009 term (quarter or semester) using a core of items 
from existing ACT surveys for two-year college students, with some edits and additions. 

Students who responded to the first survey were sent another survey in May 2010 and were 
followed with semi-annual surveys through Spring 2012. The type of survey they received at 
each juncture depended on whether they were still enrolled or had not re-enrolled at the college 
that tenn. In the subsequent two years, abbreviated versions of the Student Experience Survey 
and the Withdrawing/Nonretuming Student Survey were used, and students who completed their 
programs were surveyed to understand their post-college experiences. (The Cohort 2 Student 
Experience Survey is provided in Appendix A.) Overall, 1,495 students were surveyed: 236 from 
Cohort 1 and 1,259 students from Cohort 2. Students who returned a survey provided permission 
to link their survey responses to their de-identified institutional data, including courses taken and 
grades, so that these two data sources could be used to supplement and contextualize each other. 

Surveys were administered electronically. Students were initially sent an e-mail with a link to the 
survey and a cover note from a college administrator at each site. This email was followed by 
reminder emails and phone calls to non-respondents. Respondents received a $20 gift card for 
completing each survey. Despite this incentive, the response rate to the first administration of the 
Cohort 1 survey was quite low, ranging from 18.4% at Freedom County to 28.2% at Valley 
West. The overall response rate was 23.3% and became even lower over subsequent 
administrations. We therefore decided to take a different approach for Cohort 2. First, we used 
the Cohort 1 survey results to identify items that would be of greater use on the Cohort 2 survey. 
Then, surveys were handed out in classrooms as paper forms that were completed and returned 
during the class period. The Cohort 2 surveys proved to be of the greatest use for statistical 
analysis. Non-respondents constituted missing data for the survey. 

We were particularly interested in two variables included on the survey: career integration (a 
construct first described in Hirschy, Bremer, and Castellano, 2011) and stressful life events, 
including concern about one’s ability to pay tuition. 

Interview data. Interviews with administrators, staff, and students provided context for the 
institutional and survey data collected for this study. Most interviews with college staff were 
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conducted with individuals rather than groups, and involved two researchers, with one serving as 
note-taker. If the interviewee was willing, audio recordings were made in order to improve the 
quality of our interview notes. 

Interviews of college administrators and staff members were conducted in 2009 and 2010. These 
interviews provided infonnation about college philosophies, data systems, and the specifics of 
each college’s retention programs. Those selected to be interviewed had knowledge specific to 
the needs of this study and were identified to us by our primary contact at each site. 

Interviews of students took place during site visits in the 2011-2012 academic year. These 
interviews provided infonnation about how students decided to attend college, how they chose a 
major, how they learned about the retention services and programs available to them, and what 
kinds of experiences they had in their programs. 


IV. Data Analysis 


Analyses of the effects of retention programs using institutional data employed logistic 
regression because the dependent variables of interest (retention and completion) are 
dichotomous variables. The general fonn of the logistic regressions employed in this study is 
given by the equation: 


log 


( P(Yj= 1) \ 



ft*. 


where q is a given covariate, X qi is a value for student i on covariate q, f} is the coefficient for 
covariate q and therefore the effect of the covariate on the outcome of interest Y . , P(Y j = 1) is 
the probability of a successful outcome (i.e. student was retained) for student i and therefore 
1 - P(Y { = 1) is the probability of an unsuccessful outcome. The outcome of retention was 

observed on a semester by semester basis, therefore this outcome was analyzed through the 
logistic regression for each semester for each year of data. The outcome of completion was 
assessed with four years of data, allowing students sufficient time to complete their 
degree/certification. 


Although students were nested within the four two-year community colleges, and this may 
impose a dependence of the observations within colleges, the sample of four institutions is not 
sufficient to analyze the data using a hierarchical linear model. Therefore, the community college 
a student attends is treated as a student-level covariate reflected through three dummy-coded 
variables where the fourth community college serves as a reference group. (The dummy 
variables are not included in the tables.) This was done to statistically control for the effect of the 
college a student attends on the outcomes of interest. The inclusion of college as a covariate was 
not done in an attempt to make comparisons between colleges but merely to control for its 
potential effect on the outcomes so that a meaningful statement can be made about the 
relationship between use of programs and services and outcomes. 


College data also varied by site in that two sites (Freedom County and River Port) used 
semesters and two (Scenic Hills and Valley West) used quarters. 
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Retention data were calculated at three points in time: retention into the second tenn of the first 
academic year, retention into the first term of the second academic year, and (for both Cohorts at 
Freedom County and only Cohort 1 at the other three colleges) retention into the first term of the 
third academic year. (Freedom County provided additional data following the completion of the 
study, which allowed inclusion of both of their cohorts at the third retention point.) Two colleges 
used the semester system (Freedom County and River Port whereas two use the quarter system 
(Scenic Hills and Valley West), thus the second term was spring for two of the colleges and 
winter for the other two. 

Demographic and student characteristic variables included in our analyses are shown in Tables 
1.1, 1.2, and 1.3 and described below. 

Student demographics. Race was treated as a binary variable: White and non- White. All 
students not classified as White or as White/non-Hispanic (this varied by campus) were 
combined in a single category due to low numbers in individual subgroups. Gender was treated 
as binary. Age is continuous for most analyses (noted as Age in tables) but was treated as 
dichotomous (under 25 vs. 25+) for other analyses (noted as Age 25+ in tables). In those 
analyses where either (a) we were looking for specific interactions or (b) the construct we were 
looking for was thought to be related to younger students or older students as a group, we treated 
age as a dichotomous variable. Age 25 is frequently used as a cutoff in the literature. 

Student characteristics. Placement test scores (in reading, writing, and math) were used as 
student characteristic variables because students entered college with skill levels measured by 
these tests and this characteristic, determined at a student’s point of entry into the college, was 
not malleable by the college. We wanted to see how students with different placement scores 
fared later in college. 

Intensity of enrollment (Full-Time vs. Part-Time). Although the ratio of credits attempted to 
credits earned in the first tenn is a type of intermediate outcome, this variable is treated as a 
student characteristic, in a separate analysis, because we were interested to find out if this aspect 
of the first-term experience had an impact on future retention; it is analyzed in combination with 
full-time versus part-time enrollment in the first tenn. It should be noted that this ratio was 
highly correlated with the simple variable of credits earned. 

Occupational major was also used as a student characteristic type of variable rather than as an 
outcome variable for purposes of analysis. Students were classified as occupational if they had 
an occupational major (as defined by the individual college) at any time during the course of data 
collection. Although choosing an occupational major may be a malleable characteristic (similar 
to credits attempted/eamed and intensity of enrollment), we were more concerned with whether 
occupational and non-occupational students were differently affected by participation in 
programs and services or were more likely to be retained. 

Stress. Overall stress and pay stress, as measured on the Cohort 2 survey given in late Fall 2010, 
are also considered student characteristics. Stress was measured in a first-term survey (see 
Appendix A) of Cohort 2 students and was analyzed for respondents as two separate variables: 
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“stress” (i.e., number of stressful life events reported for the past year) and “pay stress” (i.e., 
stress specifically about paying tuition). Events included in the count of overall stress were (a) 
change or loss of job, (b) death of family member or close friend, (c) increase in family 
responsibilities, (d) financial setbacks, and (e) worry about being able to pay for classes at 
college (pay stress). 

Intermediate outcomes. GPA in non-developmental courses (calculated cumulatively 
throughout the course of the study) was an intermediate outcome considered in our analyses. 

Findings 

Overall, our analyses indicated that occupational students were more likely to be retained, as 
were students taking developmental courses in the first term, older students, female students, 
White students, and those receiving financial aid (see Table 2). White students were more likely 
than non- White students to be retained into the second term and second year (first and second 
retention points), but this effect faded with time and was no longer significant at the third 
retention point. Considering placement scores, and controlling for other variables, Math 
placement scores were predictive of retention at all three points in time. Reading placement 
scores predicted retention into the second tenn and second year, but not into the third year. 
Writing placement scores did not significantly predict retention at any point in time, though a 
weak relationship (p = .09) was observed in retention to the second term. 


Table 2: Developmental Enrollment (Term 1), PI 

lacement, and Retention 

Variable 

Retention to 2 nd term 
(A =5,060) 

Retention to Term 1 
of 2 nd year 
(A =5,060) 

Retention to Tenn 1 
of 3 ld year Cohort 1 
plus Freedom 
County Cohort 2 
(A =2,928) 

b 

P 

b 

P 

b 

P 

DERW 1 

.347 

.004 

.453 

.000 

.301 

.049 

DMath 1 

.219 

.019 

.022 

.775 

.249 

.032 

DERW lxDM 1 

-.103 

.524 

-.268 

.050 

-.350 

.082 

Reading Placement 

.111 

.023 

.122 

.005 

.069 

.284 

Writing Placement 

.084 

.090 

.015 

.730 

.049 

.448 

Math Placement 

.313 

.000 

.327 

.000 

.267 

.000 

Age 

.004 

.308 

.010 

.002 

.016 

.001 

White 

.304 

.000 

.264 

.000 

-.126 

.223 

Female 

.086 

.227 

.154 

.011 

.327 

.000 

Occupational 

.297 

.000 

.357 

.000 

.333 

.001 

Financial Aid 

1.293 

.000 

.726 

.000 

.502 

.000 
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Table 3: Descriptive Statistics for GPA, Developmental Course Enrollment, Placement 
Tests, Age, Race, Gender, Occupational Major, and Financial Aid 

Variable 

M 

SD 

N 

GPA Non-Developmental ERW/Math 

2.4389 

1.18395 

4546 

DERW 1 

.23 

.421 

4546 

DMath 1 

.39 

.487 

4546 

DERW lxDM 1 

.12 

.321 

4546 

Reading Placement 

.0614 

.94919 

4546 

Writing Placement 

.0701 

.95909 

4546 

Math Placement 

.0362 

.99263 

4546 

Age 

25.03 

9.345 

4546 

White 

.65 

.478 

4546 

Female 

.51 

.500 

4546 

Occupational 

.59 

.491 

4546 

Financial Aid 

.74 

.438 

4546 


Occupational students, older students, female students, White students, students receiving 
financial aid, and students taking developmental math in the first tenn had higher GPAs in non- 
developmental courses (see Table 4). Scores on each of the three types of placement test were 
predictive of higher GPA in non-developmental courses (see Table 4). 


Table 4: Developmental Enrollment (Term 1), Placement, and Nondevelopmental 
GPA 

Variable 

b 

SE 

P 

P 

DERW 1 

0.072 

0.060 

0.025 

0.233 

DMath 1 

-0.178 

0.044 

-0.073 

0.000 

DERW lxDM 1 

0.070 

0.079 

0.019 

0.376 

Reading Placement 

0.056 

0.024 

0.045 

0.022 

Writing Placement 

0.065 

0.024 

0.053 

0.008 

Math Placement 

0.218 

0.020 

0.183 

0.000 

Age 

0.021 

0.002 

0.162 

0.000 

White 

0.338 

0.039 

0.136 

0.000 

Female 

0.202 

0.034 

0.085 

0.000 

Occupational 

0.141 

0.037 

0.059 

0.000 

Financial Aid 

0.119 

0.040 

0.044 

0.003 


Both overall stress and pay stress were predictive of non-retention (see Tables 5, 6). 
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Table 5: Overall Stress and Retention 

Variable 

Retention 

(N = 

to 2 nd tenn 

L ,02 1) 

Retention t< 
2 nd , 
(N = 

D Term 1 of 
/ear 

L ,02 1) 

Retention to Tenn 1 of 
3 ld year (Cohort 1 only) 
(A =648) 

b 

P 

b 

P 

b 

P 

Stress mean 

-.789 

.013 

-.964 

.000 

-.790 

mi 

Financial Aid 

.516 

.214 

-.264 

.362 

-.532 

.101 

StressxFinAid 

.568 

.120 

.781 

.008 

.521 

.173 

Occupational 

-.337 

.260 

-.107 

.529 

-.319 

.131 

Female 

.004 

.595 

.000 

.473 

.000 

.933 

White 

-.090 

.738 

.297 

.057 

.606 

.012 

Age 25+ 

-.182 

.472 

-.218 

.163 

.140 

.534 


Table 6: Stress about Paying Tuition and Retention 


Retention 
(A = 1 

to 2 nd term 

L ,03 1) 

Retention t< 
2 nd : 
(N= i 

a Term 1 of 
year 

1,010) 

Retention to Term 1 of 
3 rd year (Cohort 1 only) 
(A =655) 

Variable 

b 

P 

b 

P 

b 

P 

Pay Stress 

-.703 

.000 

-.705 

.000 

-.085 

.697 

Financial Aid 

.606 

.097 

-.191 

.433 

-.052 

.847 

Pay StressxF inAid 

.405 

.071 

.628 

.001 

-.175 

.462 

Occupational 

-.111 

.357 

-.050 

.766 

-.281 

.181 

Female 

.004 

.602 

.000 

.491 

.000 

.972 

White 

-.099 

.716 

.281 

.073 

.662 

.006 

Age 25+ 

-.183 

.473 

-.274 

.078 

.083 

.706 


Financial Aid. Students who ever received financial aid (when measured as a cumulative, 
dichotomous variable across all terms) were more likely to be retained at each retention point. 
Students receiving financial aid had higher GPAs in non-developmental courses. Most students 
(68%) who received financial aid began receiving it during their first term. When controlling for 
Occupational major, race, sex, age, and placement scores, and looking at types of financial aid by 
category, students receiving Pell grants, Stafford subsidized loans, and Other types of financial 
aid were retained at significantly higher levels than similar students not receiving financial aid, 
whereas those receiving unsubsidized Stafford loans were not retained at a significantly higher 
level than those receiving no financial aid (see Table 7). Controlling for placement scores did not 
appreciably alter these results. 
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Table 7: Financial Aid Only 

Variable 

Retention to 2 nd 
tenn 

(A = 7,898) 

Retention to 
Term 1 of 2 nd 
year 

(A = 7,898) 

Retention to Tenn 1 of 
3 ld year Cohort 1 plus 
Freedom County 
Cohort 2 
(A = 3,949) 

b 

P 

b 

P 

b 

P 

Pell 

.636 

.000 

.160 

.002 

.269 

.001 

Stafford Subsidized Loan 

.527 

.000 

.275 

.000 

.234 

.060 

Stafford Unsubsidized Loan 

.166 

.098 

.161 

.050 

.310 

.015 

Other FinAid 

1.287 

.000 

1.038 

.000 

.711 

.000 


Developmental Education. We divided developmental courses into Developmental 
Mathematics (DM) and Developmental English, reading, or writing (DERW) courses. Twenty 
percent had enrolled in DERW and 29% had enrolled in DM (see Table 1). Students who took 
DM were more likely to be retained into the second term of the first year and the first term of the 
third year, but not into the first tenn of the second year, than students who did not take DM (see 
Table 2). This was true even without accounting for placement test scores. Students who took 
DERW were retained at a significantly higher rate at each of the three points in time. 


Tutoring. Tutoring data were available from three colleges. Students who received tutoring in 
their first term at college were more likely to be retained at all three retention points (see Table 
8). This is true even though we did not adjust the results for college readiness as measured by 
placement test scores. These data suggest that students who avail themselves of tutoring are more 
likely to be retained. 


Table 8: Tutoring in First Term and Retention 


Variable 

Retention to 2 nd 
term 

(N =3,950) 

Retention to Term 1 
of 2nd year 
(A =3,950) 

Retention to Tenn 1 of 
3rd year (Cohort 1 
only) 

(A =2,348) 

b 

P 

b 

P 

b 

P 

DERW 1 

.313 

.020 

.494 

.000 

.222 

.187 

DMath 1 

.175 

.118 

.069 

.470 

.098 

.476 

DERWxDMath 

-.201 

.275 

-.462 

.003 

-.394 

.081 

Tutoring 1 

.841 

.000 

.578 

.000 

.488 

.000 

Reading Placement 

.113 

.051 

.134 

.009 

.116 

.113 

Writing Placement 

.073 

.198 

.012 

.813 

.006 

.930 

Math Placement 

.300 

.000 

.344 

.000 

.286 

.000 

Age 

-.001 

.744 

.007 

.064 

.014 

.007 

White 

.377 

.000 

.361 

.000 

-.005 

.968 

Female 

.092 

.255 

.167 

.015 

.344 

.001 

Occupational 

.298 

.001 

.332 

.000 

.430 

.000 

FinAid 

1.368 

.000 

.812 

.000 

.799 

.000 


21 




Intensity of enrollment and ratio of credits earned to credits attempted. Students who 
enrolled full-time in the first semester showed higher retention rates than those enrolling part- 
time. Students who completed all of the credits they enrolled in were more likely to be retained 
(see Table 9). The value of completing all credits attempted in the first tenn greatly exceeded the 
value of enrolling full-time in the first tenn (Figures 3 a and 3 b). 


Table 9: Retention and Credits Attempted/Credits Earned 

Retention to First Term of Second Year 

Credit Status 

Full Time 
(%) 

Part Time 
(%) 

Completed all credits attempted 

74 

64 

Completed some credits attempted 

46 

31 


Retention to First Term of Third Year 

Credit Status 

Full Time 
(%) 

Part Time 
(%) 

Completed all credits attempted 

51 

42 

Completed some credits attempted 

26 

18 



FIGURE 3 a. Retention to first tenn of second year. 
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Interactions 

In order to determine what worked for whom, we examined several interactions to determine 
whether some groups of students were differently affected by participation in programs and 
services. Our literature review identified a number of interactions related to the variables under 
consideration. In particular, age may moderate the effect of developmental education (Jepsen, 
2006). Our results related to this interaction were mixed. Overall, older students had better 
retention at the second and third retention points, but there was no significant difference at the 
first retention point. For retention at the first and third retention points, there was no interaction 
between DM and age. DM appeared useful for older students at the second retention point. 
DERW was associated with better retention for younger students at the first two retention points. 
The literature also suggested that college readiness (as measured by placement scores) may 
moderate the effect of tutoring (Maxwell, 1990), and that race may moderate the effect of type of 
financial aid (see Table 10). Our data did not provide support for either of these possibilities. 

In analyzing the interaction between race and financial aid, each of which aided retention on its 
own, we found that these were essentially independent in their impact on retention, with no 
significant interaction (see Table 10). This was true regardless of whether we used total financial 
aid or its constituent types. 
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Table 10: Race, Financial Aid, and Retention 

Variable 

Retention to 2 nd term 
(A = 10,948) 

Retention to Term 1 of 
2 nd year 
(A = 10,948) 

Retention to Term 1 of 
3 rd year (Cohort 1 
only) 

(A =7,248) 

b 

P 

b 

P 

b 

P 

White 

.590 

.000 

.415 

.000 

.410 

.010 

Financial Aid 

1.270 

.000 

.562 

.000 

.458 

.004 

WhitexFinAid 

-.174 

.110 

.032 

.111 

-.057 

.739 

Occupational 

.165 

.001 

.213 

.000 

.175 

.003 

Female 

.000 

.609 

.000 

.990 

.000 

.384 

Age 25+ 

-.246 

.000 

-.134 

.001 

-.096 

.092 


The interaction between occupational major and financial aid also failed to reach significance 
(see Table 1 1). 


Table 11: Occupational Status, Financial 

1 Aid, and Retention 

Variable 

Retention to 2 nd term 
(A = 10,948) 

Retention to Term 1 of 
2 nd year (A = 10,948) 

Retention to Tenn 1 of 
3 rd year (Cohort 1 only) 
(A =7,248) 

b 

P 

b 

P 

b 

P 

Occupational 

.247 

.001 

.332 

.000 

.323 

.005 

Financial Aid 

1.229 

.000 

.696 

.000 

.549 

.000 

OccxFinAid 

-.136 

.170 

-.172 

.062 

-.201 

.129 

Female 

.000 

.610 

.000 

.993 

.000 

.386 

Age 25+ 

-.246 

.000 

-.135 

.001 

-.096 

.091 

White 

.464 

.000 

.438 

.000 

.360 

.000 


There was no interaction between occupational major and race (see Table 12) or occupational 
major and gender (see Table 13). 


Table 12: Occupational Status, Race, and Retention 

Variable 

Retention to 2 nd term 
(A= 10,948) 

Retention 
of 2 nd 
(A= 11 

to Term 1 
year 

3,948) 

Retention to Tenn 1 of 
3 ld year (Cohort 1 only) 
(A =7,248) 

b 

P 

b 

P 

b 

P 

Occupational 

.305 

.000 

.190 

.012 

.215 

.066 

White 

.259 

.001 

.262 

.000 

.295 

.011 

Occupationalx White 

-.005 

.959 

.125 

.167 

.007 

.961 

Female 

.000 

.495 

.000 

.924 

.000 

.372 

Age 25+ 

-.227 

.000 

-.128 

.002 

-.087 

.124 
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Table 13: Occupational Status, Sex, and Retention 

Variables 

Retention to 2 nd tenn 
(N= 10,948) 

Retention to Term 1 of 
2 nd year 
(N= 10,948) 

Retention to Term 1 of 
3 ld year (Cohort 1 only) 
(N = 7,248) 

b 

P 

b 

P 

b 

P 

Occupational 

.305 

.000 

.278 

.000 

.217 

.000 

Female 

.001 

.275 

.000 

.795 

.000 

.993 

OccxFemale 

-.001 

.333 

.000 

.811 

.000 

.639 

Age 25+ 

-.226 

.000 

-.129 

.002 

-.088 

.121 

White 

.256 

.000 

.349 

.000 

.300 

.000 


Considering occupational major and math placement (see Table 14), there was a significant 
interaction for retention at the third retention point (beginning of third year); having both an 
occupational major and higher math placement was predictive of retention (p = .018). 


Table 14: Occupational Status, Math Placement, and Retention 

Variables 

Retention to 2 nd 
tenn 

(N= 9,171) 

Retention to Term 

1 of 2 nd year 
(N= 9,171) 

Retention to Term 

1 of 3 ld year 
(Cohort 1 only) 

(A =6,276) 

b 

P 

b 

P 

b 

P 

Occupational 

.274 

.000 

.265 

.000 

.174 

.006 

Math Placement 

.227 

.000 

.250 

.000 

.058 

.261 

OccupationalxMath Placement 

-.047 

.378 

-.028 

.535 

.144 

.018 

Female 

.001 

.305 

.000 

.919 

.000 

.337 

Age 25+ 

.011 

.843 

.051 

.287 

.054 

.397 

White 

.296 

.000 

.394 

.000 

.375 

.000 


There was an interaction between age and occupational major (see Table 15). 


Table 15: Occupational Status, Age, and Retention 

Variables 

Retention i 
(N= 1 

to 2 nd tenn 
0,948) 

Retention tc 

2 nd i 

(N= l" 

r Term 1 of 
/ear 

0,948) 

Retention to Term 1 of 
3 ld year (Cohort 1 only) 
(N= 7,248) 

b 

P 

b 

P 

b 

P 

Occupational 

.166 

.003 

.204 

.000 

.2X1 

.001 

Age 25+ 

-.557 

.000 

-.346 

.000 

-.096 

.450 

OccxAge 25+ 

.468 

.000 

.288 

.003 

.011 

.937 

Female 

.000 

.536 

.000 

.951 

.000 

.373 

White 

.260 

.000 

.351 

.000 

.301 

.000 
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At the first two retention points, it helped to be both an occupational student and young. At the 
third retention point, older students had caught up to younger students. At this third retention 
point, there was no interaction between age and occupational status. Math placement and age 
each affected retention individually, but there was no significant interaction between these 
variables (see Table 16). 


Table 16: Math 1 

Placement, Age, and Retention 

Variables 

Retention to 2 nd term 
(N= 9,171) 

Retention to Tenn 1 
of 2 nd year 
(N= 9,171) 

Retention to Term 1 of 
3 rd year (Cohort 1 only) 
(A =6,276) 

b 

P 

b 

P 

b 

P 

Math Placement 

.197 

.000 

.213 

.000 

.147 

.000 

Age 25+ 

.036 

.532 

.090 

.061 

.075 

.240 

MathxAge 25+ 

-.040 

.507 

.036 

.466 

.030 

.649 

Female 

.001 

.285 

.000 

.880 

.000 

.360 

White 

.291 

.000 

.390 

.000 

.383 

.000 


Considering stress variables, students with lower overall stress and with lower pay stress were 
more likely to be retained (see Tables 5, 6) We looked at whether financial aid alleviated stress 
and pay stress and found that financial aid significantly reduced the negative retention impact of 
both types of stress, and completely counteracted the impact of pay stress on retention (see 
Tables 5 and 6). This analysis was conducted using a single financial aid variable. 

Discussion 

Notably, the higher retention rate for occupational students found in this study stands in contrast 
to data from the Beginning Postsecondary Students Longitudinal Study, which showed a higher 
persistence rate for academic (transfer) students (57%) than for CTE students (51%; National 
Center for Educational Statistics, n.d.). We speculate that our finding may be partly attributable 
to occupational students having clearer goals for attending college, or their programs being more 
cohort-like and offering fewer choices than other programs. 

When examining placement scores, math ability as measured at the time of college entrance is a 
powerful predictor of student success. Although it should not be surprising that students with 
higher prior math ability do better in college (see Attewell, Lavin, Domina, & Levey, 2006), the 
lack of a similar effect for prior reading or writing skills is of interest. The sheer breadth of 
measures on which students of higher math ability did better is remarkable: Math placement 
scores predicted better retention at all time periods, higher cumulative GPA for all non- 
developmental courses, and better performance in non-developmental math classes. Surprisingly, 
math placement scores also predicted higher GPA in non-developmental English courses (i.e., 
those counting toward a degree or credential); notably, writing placement scores did not. 

Students with higher writing placement scores showed only one benefit — a higher cumulative 
GPA for all non-developmental courses. Unsurprisingly, grades were helped by better prior 
writing skills, but students with better prior writing scores were no more likely to be retained or 
to complete their programs than other students. 
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Analyses were conducted adjusting for ability. We chose to adjust for initial ability in order to 
see the impact of developmental course-taking and other variables of interest on changes in 
outcomes, holding all other variables constant. Using this regression approach, participation in 
DERW classes in the first term contributed to next-tenn and second-year retention. However, 
participating in DM classes in the first term had no such effect. Participating in DM classes 
negated the positive effect of taking DERW classes if both classes were taken in the first tenn. 
Participating in DM classes was also negatively associated with both cumulative GPA and 
graduation. Because we controlled for initial ability as well as all other variables, this finding is 
especially noteworthy. Other researchers have found that enrollment in developmental 
coursework varies in usefulness based on student characteristics (e.g., Boatman & Long, 2010) 
and our findings do not refute theirs. We did not have sufficient sample size to examine only 
students near placement cutoffs or to consider students needing the most remediation as a 
separate group. 

Financial aid has previously been found to contribute to college retention and completion 
(Kennamer, Katsinas, & Schumaker, 2011; Prince & Jenkins, 2005), and our findings supported 
this. Students who received financial aid were more likely to be retained. When broken out by 
type of financial aid, however, those receiving unsubsidized loans did not share in this benefit, 
suggesting that the interest rate charged on a loan may have an impact on student persistence. 
Given current national policy discussions regarding student loan subsidies, this result suggests 
that careful attention should be paid to how variations in student loans may support or undermine 
our significant national investment in postsecondary education. 

We also found that older students, White/non-Hispanic students, women, and students with 
occupational majors were more likely to be retained. Lacking data to detennine the reasons for 
this, we can only speculate that these findings may reflect the impact of other unmeasured but 
correlated variables, including parents’ level of education, clarity of goals for attending college, 
variations in instruction or structure of occupational programs versus non-occupational 
programs, and/or availability of short-term or emergency financial support from employment or 
from students’ families. 

Participating in tutoring services during the first term in college was surprisingly useful to 
students throughout their college careers. Those who chose to use their college’s tutoring service 
during their first term in college had better retention into the second term, second year, and third 
year, and had higher cumulative and non-developmental English, reading, and writing GPAs. As 
with other analyses, this analysis adjusted for ability and other covariates, so this finding seems 
to support the value of tutoring services — a topic little-studied in prior higher education research. 
However, our data did not allow us to determine whether the apparent benefits of tutoring were 
due to the tutoring itself, to the interpersonal connection created by the tutoring relationship, or 
to the individual characteristics of students who chose to utilize these services. 

Although the use of placement tests has been questioned (e.g., Hughes & Scott-Clayton, 2011), 
our results indicate that placement test scores, particularly in math, are predictive of retention 
and GPA in non-developmental English, reading, and writing and math coursework, and 
developmental coursework has some value in promoting retention. We found math placement 
testing a more useful predictor of retention than reading placement, with writing placement 
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showing little value for this purpose. The utility of reading placement as a predictor and 
enrollment in DERW classes were both limited to early retention (into the second year). Neither 
DERW nor DM classes helped students’ GPA in non-developmental classes in the same 
discipline. Financial aid and tutoring were much more clearly related to student success than 
developmental coursework. Community colleges may wish to consider more factors in deciding 
which students need to take developmental coursework, offer more pre-assessment infonnation 
to students about how placement testing is used, provide more re -testing opportunities, or offer 
more specialized remediation based on placement subscales or student majors. 
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Appendix A: Cohort 2 Student Experience Survey 


STUDENT EXPERIENCE SURVEY 
Please use a check (R) to mark your answers. 


A. General Background 


1. Is English your first or primary language? 

□ Yes 

□ No 

2. Please mark all of the following people who you live with. (Mark all that apply.) 

□ I live alone 

□ Spouse or partner 

□ Parent(s) or guardian(s) 

□ Friend(s) or roommate(s) 

□ Brother(s) or sister(s) 

□ Child(ren) age birth to 5 

□ Child(ren) age 6-17 

□ Adult children 

□ Other relative(s) 


3. How supportive are the following people of you going to school? 



Not supportive 

Supportive 

Very supportive 

Does not apply 1 

Your spouse/partner 

□ 

□ 

□ 

□ 

Your child(ren) 

□ 

□ 

□ 

□ 

Your best friend(s) 

□ 

□ 

□ 

□ 

Your boss 

□ 

□ 

□ 

□ 

Your co-workers 

□ 

□ 

□ 

□ 


4. About how far from this college do you live, in miles? 

□ Less than 1 mile 

□ 1-5 miles 

□ 6-25 miles 

□ More than 25 miles 

5. Have you experienced any of the following during the PAST YEAR? (Mark all that 
apply.) 

□ Change or loss of job 

□ Death of family member or close friend 

□ Increase in family responsibilities 

□ Financial setbacks 

□ None of the above 
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6. Are you employed? 

□ I have a job now 

□ I do not have a job now (If you do not have a job now, please skip to Question 7) 

6a. About how many hours per week do you work? 

□ 1-10 hours per week 

□ 1 1-30 hours per week 

□ 3 1 -40 hours per week 

□ More than 40 hours per week 

6b. Do you work on-campus or off-campus? 

□ On-campus 

□ Off-campus 

□ Both on-campus and off-campus 

7. Do you have a certain career field in mind for what you want to do after college? 

□ Yes 

□ No (If you do not have a career field in mind, skip to Section B: Educational Background) 

7a. How related is your current job to your planned career? 

□ Very related 

□ Kind of related 

□ Not related 

□ I do not have a current job 

7b. For the following items, choose Yes or No: 



Yes 

No 

I have participated in a work-related learning experience 
(internship, cooperative work experience/CWE, etc.) 

□ 

□ 

Guest speakers have come to one or more of my college classes 
to talk about their jobs in my planned career. 

□ 

□ 

I have talked to people who have the kind of career I want. 

□ 

□ 

I have talked with local employers in my future career field. 

□ 

□ 

I have been to a job fair for my planned career. 

□ 

□ 

I have had opportunities to network with professionals in the 
career I want. 

□ 

□ 


I have visited a workplace in my planned career area. □ □ 


7c. How much do you agree with the following statements about your planned career? 



Strongly 

disagree 

Disagree 

Agree 

Strongly 

agree 

The career I am planning seems like a 
good fit for me. 

□ 

□ 

□ 

□ 

This kind of work will be satisfying to 
me. 

□ 

□ 

□ 

□ 

I understand what kinds of skills and 
knowledge I need to be successful in this 
career. 

□ 

□ 

□ 

□ 

I expect to be successful in this career. 

□ 

□ 

□ 

□ 
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1. In high school, mark whether you were primarily an: 

□ A student 

□ B student 

□ C student 

□ D student 

□ F student 

□ Don’t know 

□ Does not apply 

2. What is the highest level of education you have completed? 

□ High school diploma 

□ GED 

□ Some college courses 

□ Vocational or technical certificate 

□ Associate or other 2-year degree 

□ Bachelor’s or other 4-year degree 

□ Master’s, doctoral, or professional degree 

□ None of the above 

3. Which of the following was true for you when you first came to this college? 

□ Came directly from high school 

□ Came after working for a period of time (excluding summer work) 

□ Came after a period of caring for my family full-time 

□ Came after serving in the military 

□ Transferred from another 2-year college 

□ Transferred from a 4-year college 

□ Other 

4. How important was each of the following for you when you decided to enroll at this 


college? 






Not 

important 

Kind of 
important 

Important 

Very 

important 

To meet requirements for my 
planned career 

□ 

□ 

□ 

□ 

To advance in my current job 

□ 

□ 

□ 

□ 

To keep up with requirements of 
my current job 

□ 

□ 

□ 

□ 

To make more money 

□ 

□ 

□ 

□ 

My family wanted me to go back 
to school 

□ 

□ 

□ 

□ 

My high school teacher or 
counselor encouraged me 

□ 

□ 

□ 

□ 

My employer encouraged me 

□ 

□ 

□ 

□ 

To improve my English language 

□ 

□ 

□ 

□ 
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skills 

To get financial aid 

□ 

□ 

□ 

□ 

To change careers 

□ 

□ 

□ 

□ 

To get back to work after time at 
home 

□ 

□ 

□ 

□ 

To learn new skills while job 
hunting 

□ 

□ 

□ 

□ 


5. What were your goals for taking classes at this college? (Mark all that apply.) 

□ To prepare for a GED or high school diploma 

□ To get a job in my planned career field 

□ To get a license or certification 

□ To keep a license or certification 

□ To finish a vocational or technical program 

□ To finish an associate or other 2-year degree 

□ To transfer to a 4-year college 

5a. How important is it to you to achieve ALL of the goals you marked above? 

□ Not important 

□ Kind of important 

□ Important 

□ Very important 


6. How important were the following in your decision to come to this college? 



Not 

important 

Kind of 
important 

Important 

Very 

important 

A specific program 

□ 

□ 

□ 

□ 

How far the college is from 
where I live 

□ 

□ 

□ 

□ 

How far the college is from 
where I work 

□ 

□ 

□ 

□ 

Cost of attending the college 

□ 

□ 

□ 

□ 

Financial aid or scholarship 

□ 

□ 

□ 

□ 

Chance to work part-time on 
campus 

□ 

□ 

□ 

□ 

Entrance requirements I could 
meet 

□ 

□ 

□ 

□ 
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C. Current College Experience 


1. As a student this term, how much of a problem has each of the following been for you? 
Please mark "Does not apply" if the statement does not apply to you. 


Not a Minor 

problem problem 

Problem 

Major 

problem 

Does not 
apply 

Transportation (cost, parking, 
access to public transportation, 
etc.) 

□ 

□ 

□ 

□ 

□ 

Cost or availability of books and □ 

related materials 

□ 

□ 

□ 

□ 

Access to computer or internet 

□ 

□ 

□ 

□ 

□ 

Money problems 

□ 

□ 

□ 

□ 

□ 

Family responsibilities 

□ 

□ 

□ 

□ 

□ 

2. Are you worried about being able to pay for your classes at this college? 

□ I do not worry about paying for classes 

□ I worry a little about paying for classes 

□ I worry a lot about paying for classes 


3. For each college service or program listed below, mark your current level of satisfaction 
with it. If you have not used the service or program or if your college does not offer the 
service or program, please mark "Does not apply". 


Very 

dissatisfied Dissatisfied 

Satisfied 

Very 

satisfied 

Does not 
apply 

Academic advising or course 
planning services 

□ 

□ 

□ 

□ 

□ 

Counseling for personal 
concerns or problems 

□ 

□ 

□ 

□ 

□ 

Help with transferring to 
another college 

□ 

□ 

□ 

□ 

□ 

Career guidance and planning 
services 

□ 

□ 

□ 

□ 

□ 

Financial aid services 

□ 

□ 

□ 

□ 

□ 

Computer labs 

□ 

□ 

□ 

□ 

□ 

Library 

□ 

□ 

□ 

□ 

□ 

Tutoring or learning resource 
center services 

□ 

□ 

□ 

□ 

□ 

College orientation program 

□ 

□ 

□ 

□ 

□ 

College-sponsored social 
activities 

□ 

□ 

□ 

□ 

□ 

Cultural programs and 
activities 

□ 

□ 

□ 

□ 

□ 

Student clubs/organizations 

□ 

□ 

□ 

□ 

□ 
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Sports/athletics □ 

□ 

□ 

□ 

□ 

Disability services □ 

□ 

□ 

□ 

□ 

Student health service □ 

□ 

□ 

□ 

□ 

Student employment services □ 

□ 

□ 

□ 

□ 

On-campus Veterans services □ 

□ 

□ 

□ 

□ 

On-campus day care □ 

□ 

□ 

□ 

□ 

4. Thinking about the current term, how much do you agree with the following statements 
about your experiences at this college? 


Strongly 

disagree 

Disagree 

Agree 

Strongly 

agree 

I am satisfied with the preparation I am getting 
for my planned career. 

□ 

□ 

□ 

□ 

The classes I want to take are available at 
times I can take them. 

□ 

□ 

□ 

□ 

My interest in my planned career has grown 
since coming to this college. 

□ 

□ 

□ 

□ 

My classes are useful in helping me build the 
skills and knowledge I need for work. 

□ 

□ 

□ 

□ 

My classes will help me get a job. 

□ 

□ 

□ 

□ 

My classes will help me advance in my 
planned career. 

□ 

□ 

□ 

□ 

My classes allow me to learn hands-on what I 
need for my planned career. 

□ 

□ 

□ 

□ 

My instructors have work experience in the 
career I want, and they share that with us in 
class. 

□ 

□ 

□ 

□ 

My instructors plan events outside of class that 
help me learn more about my future career. 

□ 

□ 

□ 

□ 

5. Thinking about the current term, do you agree with the following statements about your 
experiences with other students at this college? 


Strongly 

disagree 

Disagree 

Agree 

Strongly 

agree 

I have developed good relationships with 
other students. 

□ 

□ 

□ 

□ 

Most of my friends at this college are serious 
about school. 

□ 

□ 

□ 

□ 

It has been hard for me to meet and make 
friends with other students. 

□ 

□ 

□ 

□ 

The friendships I have at this college have 
been personally satisfying. 

□ 

□ 

□ 

□ 

If I had a personal problem, I could talk about 
it with a student friend at this college. 

□ 

□ 

□ 

□ 

Most students here have values and attitudes 
that are similar to mine. 

□ 

□ 

□ 

□ 
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I know the name of at least one other student 
in each of my classes. 

□ 

□ 

□ 

□ 

I am satisfied with the racial hannony at this 
college. 

□ 

□ 

□ 

□ 


6. Thinking about the current term, do you agree with the following statements about your 


experiences with faculty and staff at this college? 


Strongly 

disagree 

Disagree 

Agree 

Strongly 

agree 

In general, I know who to go to if I need 
information. 

□ 

□ 

□ 

□ 

Most of the instructors I’ve had work hard to 
be good teachers. 

□ 

□ 

□ 

□ 

Most instructors I’ve had care about students. 

□ 

□ 

□ 

□ 

Most other college staff members I have met 
really care about students. 

□ 

□ 

□ 

□ 

I have had negative interactions with 
instructors. 

□ 

□ 

□ 

□ 

I have had negative interactions with other 
college staff. 

□ 

□ 

□ 

□ 

Most of the instructors I’ve had work hard to 
be good teachers. 

□ 

□ 

□ 

□ 

Most instructors I know at this college support 
my efforts to get a job in my planned career. 

□ 

□ 

□ 

□ 


7. If you could go back and decide again, would you choose to attend this college? 

□ Definitely yes 

□ Probably yes 

□ Probably no 

□ Definitely no 


THANK YOU! 
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